The discussion below is merely provided for general background information and is not intended to be used as an aid in determining the scope of the claimed subject matter.
Aspects of the disclosure generally relates to processing of a reference image and dynamically configuring an image processing function. In particular, though not exclusively, the disclosure relates to methods and systems for slicing a reference image and/or configuring an image processing function to process the slices of the reference image, a reference features generator, a dynamically configurable image processing module, an augmented reality device, an augmented reality system, and a computer program product using such methods.
Due to the increasing capabilities of multimedia devices, mobile augmented reality (AR) applications are rapidly expanding. These AR applications allow enrichment (augmentation) of a real scene with additional content, which may be displayed to a user in the form of a graphical layer overlaying the real-world scenery. The augmentation thereby provides an “augmented reality” user-experience.
Augmented reality platforms, such as the Layar Vision platform, allow an AR application to recognize an object in an image frame and to render and display certain content together with the recognized object. In particular, an AR application may use vision-based object recognition processes to recognize whether a particular object is present in the scene. Furthermore, the AR application may use a pose estimation process to determine position and/or orientation (pose information) of the object based on information in the image frame and sensor and/or camera parameters. The pose information is then used to generate the augmentation for the object.
Examples of known image processing algorithms for object recognition and tracking are described in the article by Duy-Nguygen Ta et al. “SURFrac: Efficient Tracking and Continuous Object Recognition using local Feature Descriptors” IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'09), Miami, Fla., Jun. 20-25, 2009. Object recognition may include extracting features from the image frame and matching these extracted features with reference features associated with objects stored in a database. By matching these reference features with the extracted features, the algorithm may determine that an object is “recognized”. Thereafter, the recognized object may be subjected to a sequential estimation process (tracking) wherein the new state of the object is estimated on the bases of new observables (e.g. a new image frame) and the previous state of the object determined on the basis of a previous image frame. The aforementioned process may be repeated for each camera image frame at a sufficient fast rate, e.g. 15 to 30 frames per second, in order to ensure that the visual output on the display is not degraded by jitter or other types of flaws.
A reference features set is created from the original reference image of the object such as the image of a full-paged magazine advertisement or the image of a billboard. Having the reference features set, the AR device can perform object recognition and tracking by matching each of a plurality of reference features sets against the extracted features set from the image captured by a camera of the AR device. In practice, object recognition and tracking work best when a sufficient amount of or sufficiently enough features of the reference features set can be matched with the features of the current camera image frame. But as soon as a user moves the camera closer to the actual object, less of the reference features set can be found matching the features in the current camera image frame. At some point, the ability to perform object recognition and/or tracking will be lost, since not enough reference features can be found in the camera image frame to allow for object recognition and/or tracking to successfully occur.
Accordingly, there is a need to provide improved methods and systems that at least alleviate some of these problems.